Maximizing Efficiency

So, I've got a DXL program that does the following:

1. Allows the user to select, in a DOORS Formal Module, one or multiple Objects before exection

2. Presents the user with a GUI asking them whether to copy in or out Links. Both checkboxes may be selected. Also allows them to select an item (or multiples) from a list box showing Absolute Number values and Object Text values of all Objects in the Module that are not selected or current.

3. On an OK button click with one or more Objects in the list selected and at least one of the checkboxes checked, copies every in/out/both Link(s) from the Object(s) chosen from the list to the Object(s) selected in the Module.

 

A use case could include a Module called "Contracts" containing Objects that represent a contract. Every requirement in the database is Linked to one or more contracts. Let's say we create a new contract, 'ContractF' and we know it will have all the Links currently contained in "ContractE" plus a few more for the new requirements associated with this contract. So, we need to copy all the Links from ContractE to ContractF and then go add the Links to the new requirements for F.

The problem with the code as it stands is that, if "ContractE" contains, say 200,000 In Links to 30 different Modules, the program will run out of RAM before it completes the job of copying all the In Links. I have included here two functions from the program, copyInLinks and createLinkForObject.

I am trying to find a way to make this code more efficient. Where am I going wrong? Any ideas?

 

About the copyInLinks function: the objList variable is a skip list containing the Objects the user selected in the GUI. These are the Objects from which the Links need to be copied. The selectedObj variable is a global Object handle that represents the current Object in a loop over a skip list of Object representing all the currently selected (or the actual current) Objects in the Module. modifiedMods is a skip list that holds all Modules that this program modifies over the course of its execution. At the end of execution they are all saved and closed.

 

/*  Function:   copyInLinks()
    Purpose:    copies the In Links from the Object the user selected in the GUI
                to the Object(s) that was(were) selected/current for the 
                Module when this program was executed
                Adds any modified Modules to the modifiedModules
                global skip list so they may later be saved.
*/
void copyInLinks()
{
    LinkRef  lr        = null;
    Link     lnk       = null;
    Object   obj       = null;
    Object   sourceObj = null;
    Module   mod       = null;
    ModName_ modName   = null;
    string   sModName  = null;
    string   lModName  = null;
    
    
    for obj in objList do
    {
        /*open the Modules for which we'll need to look at Objects
          Can't use the Link type without the Modules open. 
          Need Link to get source Object*/
        for modName in obj <-"*" do
        {
            mod = edit(fullName(modName), false);
            filtering(false);
        }
                
        /*Now we can get the Links and the source Objects*/
        for lnk in obj <-"*" do
        {
            lModName = fullName(module(lnk));
            sourceObj = source(lnk);
            if(null != sourceObj)
            {
                if(createLinkForObject(sourceObj, lModName, selectedObj))
                {
                    if(null == modifiedMods)
                        modifiedMods = create();
                        
                    if(!isModuleInSkip(modifiedMods, mod))
                    {
                        put(modifiedMods, numMods, mod);
                        numMods++;
                    }//if source mod wasn't in the modifiedMods list
                }//if the Link was succesfully created
            }//if the source Object could be obtained
        }//for each In Link
    }//for each Object selected in the Module upon execution
}
/*  Function:   createLinkForObject()
    Purpose:    Creates the link from obj to objLink through
                Link Module lLinkModuleId
    Return:     Whether the creation of the Link was successful
*/
bool createLinkForObject(Object sourceObj, string lMod, Object tObj)
{
    string errMessage = null;
    
    if(null == lMod)
    {
        logError("Cannot create a Link through a null Link Module.\n");
    }
    if(null != sourceObj && null != tObj)
    {
        if(canModify(sourceObj))
        {
            noError();
            sourceObj -> lMod -> tObj;
            errMessage = lastError();
            if(null == errMessage)
                return true;
            else
                logMessage("Could not create the Link because of the following error: \n" errMessage);
        }
        else
        {
            errMessage = "Could not create the Link because Source Object " getAbsoluteNumber(sourceObj) " in Module ";
            errMessage = errMessage fullName(module(sourceObj)) " cannot be modified.";
            logMessage(errMessage);
            return false;
        }//canModify(sourceObj)
    }//null != sourceObj && null!= tObj
    else
    {
        if(null == sourceObj)
            logMessage("Could not create the Link because the source Object does not exist.\n");
        else
            logMessage("Could not create the Link because the target Object does not exist.\n");
        return false;
    }
    return false;
}

 


woodpryan - Tue Jun 16 17:25:12 EDT 2015

Re: Maximizing Efficiency
Wolfgang Uhr - Wed Jun 17 02:33:17 EDT 2015

Hi

You've loade a set of up to 30 modules at once? Ok, this will lead to a crash (It depends of the size of the modules).

Try to load one module, then perform the link copy process only for this module, save the data, unload the module and load the next.

Best regards

Wolfgang

Re: Maximizing Efficiency
DoorsXLAustralia - Wed Jun 17 10:50:24 EDT 2015

What version of DOORS are you using?

Older 32 bit versions of DOORS top out at around 3 or 4 GB.

Newer 64 bit DOORS (the newest 9.6.1.2 is 64 bit only) have no practical memory limit within the constraints of current hardware specs.

Re: Maximizing Efficiency
woodpryan - Wed Jun 17 11:07:14 EDT 2015

We are using version 9.3. It has a memory limit of 2GB. This can be changed, but getting IT to do something like that around this place is like pulling teeth.

I completely understand this problem of opening so many Modules at once, but if I close each Module after its Links have been copied the program's execution time is multiplied. I mean, we do lose a whole lot of time. Also, I cannot detect incoming Links to an Object if all of the Modules to which it is Linked are not open. That's why I open them first.

Re: Maximizing Efficiency
pommCannelle - Wed Jun 17 11:21:03 EDT 2015

( hmm ...
First of all, you don't need a skip to store anything ... copy the link when you find it is enough, why do you need to build a skip list  with 200.000 entries ?
Then, you don't need to store any trace in memory ... you are going to lost it if a crash occures anyway. I prefer to open a stream and write my reports during the execution of the script.
In this activity, to use 3 or 4 buffers as text constant and then access them using the tempStringOf function is the good thing to do ;)
You don't need to specify link module to use if your linkset is properly set ... so don't ;)
Then a simple ' if ( null ( sourceObj->targetObj) ) ...' should allows you to catch errors.
I agree with Master Wolfgang ... You can even add a counter to deal with the firsts 1000 objects ... Then the next 1000 ... 

It's a big issue .... Have fun with your memory lacks ;(

)

 

 

Re: Maximizing Efficiency
woodpryan - Wed Jun 17 11:48:04 EDT 2015

Ok, so (in psuedocode) the way this thing operates at the moment is as follows:

 

Create list of selected Objects in the current Module, including current Object

Get list of Objects use selects in the GUI

for each Object in guiSelectedList

{

    if(inLinksSelected)

    {

        for each Object in moduleSelectedList

        {

            for each source module name

                openModuleForEdit

            for each link coming in through any Link Module

            {

                get LinkModule name

                get sourceObject

                createLink(sourceObj, LinkMod, guiSelectedObj)

                if(modifiedModulesList does not contain(SourceModule))

                    put(modifiedMods, sourceMod)

            }for each Object selected from the gui

        }for each object currently selected in the current module

    }do copy in links

}next gui selected Object

saveAndCloseModifiedMods

 

If I understand correctly, you are suggesting that the logic of this program be completely rewritten as follows:

 

get list of currently selected Objects, including current Object

get list of Objects selected in the GUI

for each GUISelected Object

{

    for each source module name linked to obj

    {

        openSourceModForEdit

        for each incoming link through any Link Module

        {

            getSourceObj

            getLinkModName

            for each moduleSelectedObject

            {

                createLink(sourceObj, linkModName, moduleSelectedObject)

            }each Object currently selected in the Module

        }each incoming link

    }each source module name

    saveAndClose(sourceModule)

}each Gui selected Object

 

This may greatly increase the amount of time it takes for this program to execute, but it also may save us a lot of memory. Using this method, I think we end up opening, saving, and closing Modules a lot. Those operations are slow, which is why I didn't write it this way to begin with.

Is this the only way to decrease my RAM usage on this thing? Are there any other suggestions? The customer may be pretty unhappy about the speed decrease here.

       

   

 

Re: Maximizing Efficiency
pommCannelle - Wed Jun 17 12:22:49 EDT 2015

woodpryan - Wed Jun 17 11:48:04 EDT 2015

Ok, so (in psuedocode) the way this thing operates at the moment is as follows:

 

Create list of selected Objects in the current Module, including current Object

Get list of Objects use selects in the GUI

for each Object in guiSelectedList

{

    if(inLinksSelected)

    {

        for each Object in moduleSelectedList

        {

            for each source module name

                openModuleForEdit

            for each link coming in through any Link Module

            {

                get LinkModule name

                get sourceObject

                createLink(sourceObj, LinkMod, guiSelectedObj)

                if(modifiedModulesList does not contain(SourceModule))

                    put(modifiedMods, sourceMod)

            }for each Object selected from the gui

        }for each object currently selected in the current module

    }do copy in links

}next gui selected Object

saveAndCloseModifiedMods

 

If I understand correctly, you are suggesting that the logic of this program be completely rewritten as follows:

 

get list of currently selected Objects, including current Object

get list of Objects selected in the GUI

for each GUISelected Object

{

    for each source module name linked to obj

    {

        openSourceModForEdit

        for each incoming link through any Link Module

        {

            getSourceObj

            getLinkModName

            for each moduleSelectedObject

            {

                createLink(sourceObj, linkModName, moduleSelectedObject)

            }each Object currently selected in the Module

        }each incoming link

    }each source module name

    saveAndClose(sourceModule)

}each Gui selected Object

 

This may greatly increase the amount of time it takes for this program to execute, but it also may save us a lot of memory. Using this method, I think we end up opening, saving, and closing Modules a lot. Those operations are slow, which is why I didn't write it this way to begin with.

Is this the only way to decrease my RAM usage on this thing? Are there any other suggestions? The customer may be pretty unhappy about the speed decrease here.

       

   

 

Yes, this is almost what i had in mind ;)

I think you should consider to do the treatment module source by module source .... 
No need to open the 30 modules at the same time, no need to open them 200.000 times.

Just do the job for one module then close it and go on to the next one.
You are going to loop 30 times ... but the loop part is not the one who takes the most of your time, and this allows you to keep the needed memory contained ;)

And once again, if your linksets are configured well, you don't need to know the link module to make the link ;)

Re: Maximizing Efficiency
woodpryan - Wed Jun 17 12:30:49 EDT 2015

Ok. That makes sense. Thanks for your help. I have modified the algorithm as follows:

moduleSelectedObjects = get list of currently selected Objects, including current Object

guiSelectedObjects = get list of Objects selected in the GUI

for each GUISelected Object
{
    if(InLinksSelected)
    {
        for each source module name linked to obj
        {
            openSourceModForEdit
            for each moduleSelectedObject
            {

                for each incoming link through any Link Module
                {
                    getSourceObj
                    getLinkModName
                    createLink(sourceObj, linkModName, moduleSelectedObject)
                }each incoming link

            }each object selected in Module
        }each source module name
        saveAndClose(sourceModule)
    }if inLinks
}each Gui selected Object

I think this here is exactly what you meant, and I also think it is a much better approach. Thanks again.